Autonomous Car Decision Override

ABSTRACT

A visual display for an autonomous vehicle, comprising a first display section that displays an external view of the autonomous vehicle, a second display section that displays vehicle actions that the autonomous vehicle will take and a third display section that displays alternative vehicle actions that an authorized passenger of the autonomous vehicle may select to override the vehicle actions. A decision system for an autonomous vehicle, comprising an input sensor to collect information from passengers to determine if a passenger is an authorized passenger and a display section that displays alternative vehicle actions that an authorized passenger of the autonomous vehicle may select to override vehicle actions of the autonomous vehicle.

PRIORITY/INCORPORATION BY REFERENCE

This application claims priority to U.S. Provisional Application62/324,646 entitled “Autonomous Car Decision Override” filed on Apr. 19,2016, which is incorporated herein, in its entirety, by reference.

BACKGROUND

Currently, passengers of autonomous vehicles have no way to identify theactions the autonomous vehicle will take when a potentially hazardousobstacle, such as a pedestrian, vehicle, animal, or any other obstacle,is detected in the path of the vehicle. Furthermore, autonomous vehicleslack visual indicators to alert passengers that the vehicle is planningon changing speed, stopping, or performing another driving maneuver.This can sometimes build anxiety for the passengers because they do notknow what will happen next and also they do not have a way to change thedefault decision made by the autonomous vehicle. In short, it would beadvantageous if there was a way to indicate or convey to the passengerssitting inside the vehicle the next steps that the autonomous vehicle isgoing to take.

As autonomous vehicles evolve, there is a possibility that autonomousvehicles will not even have a steering wheel in it, so there wouldn't bean option to override autonomous vehicle's decisions. However, if theautonomous vehicle itself provides alternative decision options to thepassengers keeping the safety in mind, and thereafter, if passengerswere able to select those alternative decisions to override vehicle'sdefault decisions, then that would be really helpful, however, suchabove mentioned feature currently does not exist.

A similar situation to above is that if there was no road or a big ditchin the upcoming path of the autonomous vehicle, however, in case if theautonomous vehicle's default decision was to still proceed on that path,then currently the passengers of the autonomous self driving vehicles donot have a way of knowing what actions or next steps the autonomousvehicle would take until that action is taken. At the same time, theautonomous vehicle passengers also currently do not have a way to changethe default decision made by the autonomous vehicle. However, if thepassengers were able to select alternative decisions to overridevehicle's default decisions, then that would be really helpful, however,such above mentioned features currently do not exist in the autonomousself-driving vehicles.

According to already existing patents granted, autonomous vehicles willindicate to the outside pedestrians about the actions that theautonomous vehicle is about to take. However, there is currently nofeature within the autonomous vehicles to display the next steps oractions that the vehicle would take to the passengers sitting inside thevehicle. Also, there is currently no option for passengers to overridethe autonomous vehicle's default decisions by selecting the alternativedecision options that are presented by the autonomous vehicle itself bykeeping the safety in mind.

SUMMARY

Described is a visual display for an autonomous vehicle. The visualdisplay includes a first display section that displays an external viewof the autonomous vehicle, a second display section that displaysvehicle actions that the autonomous vehicle will take and a thirddisplay section that displays alternative vehicle actions that anauthorized passenger of the autonomous vehicle may select to overridethe vehicle actions.

Further described is a method for controlling an autonomous vehicle. Themethod includes displaying an external view of the autonomous vehicle,displaying vehicle actions that the autonomous vehicle will take,displaying alternative vehicle actions that the autonomous vehicle maytake, receiving a selection of the alternative vehicle actions from anauthorized passenger of the autonomous vehicle and overriding thevehicle actions with the selected alternative vehicle actions.

In addition, a further visual display for an autonomous vehicle isdescribed. The further visual display includes a first display sectionthat displays vehicle actions that the autonomous vehicle will take anda second display section that displays alternative vehicle actions thatan authorized passenger of the autonomous vehicle may select to overridethe vehicle actions.

A decision system for an autonomous vehicle, comprising an input sensorto collect information from passengers to determine if a passenger is anauthorized passenger and a display section that displays alternativevehicle actions that an authorized passenger of the autonomous vehiclemay select to override vehicle actions of the autonomous vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a visual display that is used to inform passengers of thedecisions made by an autonomous vehicle according to the exemplaryembodiments.

FIG. 2 shows an exemplary alternative mechanism to select alternativedecision options via a vehicle steering wheel with a button according tothe exemplary embodiments.

FIG. 3 shows an alternative exemplary embodiment to be able to selectthe alternative decision options listed on the display of FIG. 1.

DETAILED DESCRIPTION

The present invention may be further understood with reference to thefollowing description and the appended drawings, wherein like elementsare provided with the same reference numerals. The exemplary embodimentsshow a system, method and apparatus to inform passengers of thedecisions made by an autonomous vehicle, based on external conditions,and provide the passenger with the ability to select alternativedecisions, thus commanding the autonomous vehicle while the vehicleremains in autonomous driving mode.

It should be noted that the terms “decisions,” “vehicle decisions,”“actions,” and “vehicle actions” may be used interchangeably throughoutthis description. In this description, these terms are used to describethe actions that will be taken by the autonomous vehicle based on thevehicle's programming. For example, based on the external factors thatthe autonomous vehicle senses, the vehicle's software may decide to slowdown, increase speed, maintain the current speed, come to a stop, alterthe course of the vehicle, make a turn, etc. and cause the autonomousvehicle to perform the selected action. In addition, the term“alternative” may be used to modify these above terms. Where the term“alternative” is used to modify these terms, it indicates that anauthorized passenger may select a different vehicle action or decisionthat is different from the one selected by the vehicle's programming.

FIG. 1 shows a visual display 10 that is used to inform passengers ofthe decisions made by an autonomous vehicle, based on externalconditions, and provide the passenger with the ability to selectalternative decisions, thus commanding the autonomous vehicle while thevehicle remains in autonomous driving mode. The display 10 may be of anytype including but not limited to LCD, LED, or haptic displays. Thedisplay 10 may be located in any part of the vehicle, where it caneither be in the front, left, right, top, bottom, or center of thevehicle. The display 10 may be either at a single or multiple locationsin the autonomous vehicle. In one example, there may be a single display10 that is within the reach of the person in the driver's seat. Inanother example, the single display 10 may be mounted in a centerconsole such that the persons in the front seats may access the display10. In a further example, each seat of the autonomous vehicle may haveaccess to a display 10, e.g., the rear of the front seats may eachinclude a display 10 that are accessible to the persons in the backseats. Also, the display 10 may be affixed, attached, or permanentlymounted in the autonomous vehicle.

In the exemplary embodiment of FIG. 1, the display 10 is a touchscreenand comprises section 13 that continuously displays the external viewwhere the vehicle is going along with the potentially hazardous externalobstacles such as a pedestrian, vehicle, animal, ditch, or any otherobstacle identified by sensors and/or computer processing. The display10 may identify the potentially hazardous external obstacles. Theidentification of the potentially hazardous external obstacles may bevia a graphical display on the display 10, a highlight on the display10, an outline on the display 10 or any other manner of indicating tothe passengers of the vehicle that there is a potentially hazardousexternal obstacle. The external view as shown in section 13 may comprisea front view, rear view, side views, or any other view depending uponthe vehicle's movement. The external view of section 13 may alsocomprise the overlaid simulated navigation path of the autonomousvehicle to display the upcoming decisions and proceeding actions theautonomous vehicle plans to take based on sensor input from the externalenvironment. This simulated navigation path of the autonomous vehiclecan be demonstrated in a number of ways comprising either independent ora combination of lines, arrows, images, moving images, animation,symbols, or any other simulated navigation path.

The section 13 may also display any sensor malfunctions of theautonomous vehicle or if a sensor is not functioning, informing thepassenger of the vehicle's diminished capacity to make informeddecisions.

The display 10 of FIG. 1 also comprises a section 11 listing thedecisions and proceeding actions the vehicle will take based on sensorinput from the external environment. In addition to listing the nextsteps that the vehicle is about to take, section 11 may also compriseadditional visual indicators that indicate to the passengers if thevehicle is about to slow down, increase speed, maintain the currentspeed, come to a stop, make a turn, etc. The section 11 may also displayany sensor malfunctions of the autonomous vehicle or if a sensor is notfunctioning, informing the passenger of the vehicle's diminishedcapacity to make informed decisions.

In addition to listing the decisions and proceeding or next actions thevehicle will take in section 11 of display 10, there may also be visualindicators located in different parts of the vehicle that indicate tothe passengers if the vehicle is about to slow down, increase speed,maintain the current speed, come to a stop, make a turn, etc. Suchvisual indicators may be affixed, attached or permanently mounted in thevehicle at any location, wherein even if the passengers were notconstantly looking at display 10, they would still be able to quicklylook at the visual indicators located in one or multiple locationsaround the vehicle and know the next steps that the autonomous vehicleis about to take.

The display 10 of FIG. 1 also comprises a section 12 that provides foralternative decision options that can be selected by an authorizedpassenger if they want to override the default decisions listed insection 11. The alternative decision options may comprise alteringvehicle course, stopping the vehicle, slowing down the vehicle,increasing the speed of the vehicle, etc.

An authorized passenger may be anyone designated by the vehicle owner asa person approved to override decisions made by an autonomous vehicle.

An alternative to continuous display of potentially hazardous externalobstacles on section 13 of display 10 may be an inquiry function withinsection 13 itself or an external independent function that allows apassenger to inquire if the vehicle has detected a potentially hazardousexternal obstacle such as a pedestrian, vehicle, animal, ditch, etc.When the inquiry function is selected, section 13 will display theexternal obstacles the vehicle has detected, section 11 will display thedefault decisions made by the autonomous vehicle based on one or moreexternal obstacles identified, and section 12 will provide alternativedecision options for the passenger comprising of altering vehiclecourse, stopping the vehicle, slowing down the vehicle, increasing thespeed of the vehicle, etc.

A substitute for section 12 of FIG. 1 that displays a list ofalternative decision options may be that the section 13 itself displaysthe alternative decision options overlaid directly on the external viewof section 13, wherein an authorized passenger would be able to selectone or more of those alternative decision options and override thedefault decision of the autonomous vehicle. These alternative decisionoptions may be integrated or overlaid on the external view of section 13in a number of ways comprising of either independent or a combination oflines, arrows, images, moving images, animation, symbols, differentcolor, etc. and would not be limited to just the ways mentioned above aslong as those alternative decision options can be overlaid on section13.

Another substitute for section 12 of FIG. 1 that displays a list ofalternative decision options may be that the section 11 displays thealternative decision options in a new or a pop-up window when anauthorized passenger selects an existing default decision or one of itssubsets in section 11. In this case, an authorized passenger would beable to select one or more of those alternative decision options andoverride the default decision of the autonomous self-driving vehicleusing section 11.

In addition, an alternative to section 11 of FIG. 1 that displaysdefault decisions made by an autonomous vehicle, may be to overlay thosedefault decisions on the external view of section 13 itself. In thiscase, passengers would be able to view the default decisions made by theautonomous vehicle directly on section 13.

FIG. 1 also shows a knob 14 in close proximity to display 10. The knob14 provides the authorized passenger with the ability to selectalternative decision options displayed in section 12 if display 10 is anon-touch type screen or as an alternative to using a touch sensitivedisplay 10. In case if the alternative decision options are displayed insection 11 or section 13, then the knob 14 would still provide theauthorized passenger with the ability to select alternative decisionoptions in those sections.

FIG. 2 also shows an alternative mechanism to select alternativedecision options via a vehicle steering wheel with a button 21 thatprovides for voice operated functionality and allows the authorizedpassenger in the autonomous vehicle to select an alternative decisionfrom section 12 of FIG. 1 via the passenger's voice, as received fromvarious microphone sensors throughout the vehicle. The autonomousvehicle recognizes only the voice of authorized passengers. Theauthorized passenger's voice is recognized by the autonomous vehicle andthe vehicle reacts accordingly. The passenger may select alternativedecision options from section 12 in FIG. 1 via a voice command byholding button 21. It should be noted that, as described above, thevoice input may be triggered by a button 21. However, there may be othermanners of triggering the voice input. For example, the voice input maysimply be triggered by an authorized passenger starting to speak and thevoice input recognizing the authorized passenger's voice. In anotherexample, the passenger may say a keyword to activate the decisionsystem. The keyword may be, for example, the name of the system, akeyword programmed by the passenger, a keyword programmed by the vehiclemaker, etc.

In case the alternative decision options are displayed in either section11 or section 13 of FIG. 1 rather than section 12 of FIG. 1, then alsoholding the button 21 of FIG. 2 would allow the authorized passengers inthe autonomous vehicle to be able to select those alternative decisionoptions using the voice command. One or ordinary skill in the art wouldrecognize that in order to command the vehicle, the authorizedpassengers of an autonomous vehicle can initiate their voice commands inmany different ways without departing from the spirit and scope of thepresent invention and therefore would not be limited to any of theexamples given above.

FIG. 2 also shows an example for adding an authorized passenger so thatthe vehicle will recognize the passenger's voice, a fingerprint scanningsensor 22. An owner of the vehicle would be asked to set up theirauthorization using the fingerprint scanning sensor 22 when they firstpurchase the autonomous vehicle. For example: by using scanning sensor22, an owner would add their fingerprint to the vehicle's onboardprocessing unit, making them an authorized passenger. If the ownerwants, then they would also be able to add additional authorizedpassengers and allow those passengers to also add their fingerprints tothe onboard processing unit. Any of the authorized passengers of theautonomous vehicle would be able to pre-record their voice so theautonomous vehicle recognizes their voice later when the voice controlis activated. It should be noted that any biometric sensor may besubstituted for the fingerprint scanning sensor 22, e.g., a handprintsensor, a retina scan sensor, etc.

One or ordinary skill in the art would recognize that a fingerprintscanning sensor 22 can be placed any part of the autonomous vehicle andcan be located at either single or multiple locations without departingfrom the spirit and scope of the present invention.

Authorized passengers would be able to use their voice to command thevehicle such as select alternative decision options, inquire if thevehicle has detected potentially hazardous external obstacles, etc. Inother words, once a fingerprint has been stored, the authorizedpassenger may command the vehicle via voice button 21 and verify theiridentity as an authorized passenger via scanning sensor 22 or throughvoice recognition. Alternatively, an authorized passenger may alsoselect alternative decision options and operate display 10 via thedirectional pad button 23. The button 23 provides for full operation ofdisplay 10 of FIG. 1 and allows the authorized passenger to selectalternative decision options in section 12 of FIG. 1 to command thevehicle.

FIG. 3 shows an alternative exemplary embodiment to be able to selectthe alternative decision options listed in the section 12 on display 10of FIG. 1. Autonomous vehicles do not require a steering wheel, and assuch voice activation buttons can no longer be centrally located on thesteering wheel in vehicles without a steering wheel. FIG. 3 shows adashboard 30 with a centrally located display 10 and various voicecontrol buttons 31. The voice control button 31 provides the samefunctionality of button 21 in FIG. 2. Pressing button 31 would allow theauthorized passenger in the autonomous vehicle to select an alternativedecision from section 12 of FIG. 1 via the passenger's voice, asreceived from various microphone sensors throughout the vehicle. Theautonomous vehicle recognizes only the voice of authorized passengers.The authorized passenger's voice is recognized by the autonomous vehicleand the vehicle reacts accordingly. The passenger may select alternativedecision options from section 12 in FIG. 1 via a voice command byholding button 31.

An alternative to authorizing a passenger of an autonomous vehicleinstead of a fingerprint scanner can be a facial recognition,optical/retinal recognition, ear shape recognition, other biometricrecognition, or verbal or hand-entered passcode verification.Additionally, a passenger who enters a passcode or inserts the vehiclekey would be allowed to view the alternate decision options as shown insection 12 of FIG. 1 on their personal digital screen, which would bewithin reach of their seat in the autonomous vehicle. Thus, any type ofinput sensor may be used to verify an authorized passenger. One ofordinary skill in the art would recognize that authorizing passengers ofan autonomous vehicle can be done in many different ways as long as thatauthorization allows the passenger to command the autonomous vehicle.Therefore, any of such authorization methods would be within the spiritand scope of the present invention and would not be limited to theexamples provided above.

An alternative to requiring an autonomous vehicle's passenger to beauthorized before they can override the vehicle's default decisions canbe that a particular seat in the vehicle can be assigned as theauthorized seat, wherein the passenger that sits on that authorized seatwould have access to review, inquire or especially override theautonomous vehicle's default decision. Similar to a driver seat in thetraditional non-autonomous vehicles, there can be an authorized driverseat in the autonomous vehicle, wherein such passenger sitting on thatauthorized seat would automatically have the options available to beable to override the autonomous vehicle's default decisions. In thiscase, there would be no special requirement to authorize the passengerusing mechanisms like a fingerprint scanner, facial recognition,password, or other authorization method before the passenger couldreview, inquire or override the autonomous vehicle's default decision.

An alternative to requiring an autonomous vehicle's passenger to beauthorized before they can override the vehicle's default decisions canbe that any of the passengers sitting in the autonomous vehicle would beallowed to override the vehicle's default decisions i.e. any passengercould select alternative decision options so they can override thevehicle's default decisions. In this case, there would be no specialrequirement to authorize the passenger using mechanisms like fingerprintscanner, facial recognition, password, or other authorization methodbefore the passengers could review, inquire or override the autonomousvehicle's default decision.

The exemplary embodiments may also comprise an artificial intelligencemodule, wherein the vehicle would learn the common alternative decisionoptions that the authorized vehicle passenger usually selects underdifferent manual intervention situations, and based on that artificialintelligence knowledge, system would quickly present those commonlyselected alternative decision options to the users for future similarsituations. Thus, the system would continue to learn and help thevehicle passengers with the future selection of their alternativedecision options.

The above mentioned artificial intelligence module would also learn fromthe repeated actions of the authorized passenger based on differentsituations and accordingly alter the default decisions of the autonomousvehicle, so in future similar situations, an authorized passengerwouldn't have to do a manual intervention each time by selecting analternate decision option. An example of this learning would be: if theauthorized passenger, after seeing a child on the road, always selectsthe alternative decision option to stop the vehicle at a distance, thenthe artificial intelligence module of the autonomous vehicle would learnsuch behavior and accordingly alter the default decisions of theautonomous vehicle for similar future situations to meet the passenger'spreference.

Any of the learning captured by artificial intelligence module wouldstill consider safety of the passengers and the pedestrians as thehighest priority and would not present an alternative decision option tothe passengers that could impact or jeopardize safety. In section 12 ofFIG. 1, any of the alternative decision options provided would always besuch that they consider safety of the passengers and the pedestrians asthe highest priority and would not present an alternative decisionoption to the passengers that could impact or jeopardize safety.

It will be appreciated by those skilled in the art that variousmodifications and alterations of the exemplary embodiments can be madewithout departing from the scope of the appended claims. Some of thesehave been discussed above and others will be apparent to those skilledin the art.

What is claimed is:
 1. A visual display for an autonomous vehicle,comprising: a first display section that displays an external view ofthe autonomous vehicle; a second display section that displays vehicleactions that the autonomous vehicle will take; and a third displaysection that displays alternative vehicle actions that an authorizedpassenger of the autonomous vehicle may select to override the vehicleactions.
 2. The visual display of claim 1, wherein the first displaysection further identifies potential hazardous external obstacles basedon the external view.
 3. The visual display of claim 1, wherein thefirst display section further displays overlaid simulated navigationdirections based on the vehicle actions that the autonomous vehicle willtake.
 4. The visual display of claim 1, wherein the vehicle actions arebased at least in part on sensor readings of external obstacles.
 5. Thevisual display of claim 1, wherein one of the first display section orsecond display section displays one of vehicle malfunctions or sensormalfunctions.
 6. The visual display of claim 1, wherein the visualdisplay is a plurality of visual displays mounted within the autonomousvehicle.
 7. The visual display of claim 1, wherein the external viewcomprises one of a front view, a rear view or a side view of theautonomous vehicle.
 8. The visual display of claim 1, wherein thealternative vehicle actions comprise one of slowing down, increasingspeed, maintaining the current speed, stopping, altering a course of thevehicle or making a turn.
 9. The visual display of claim 1, wherein atleast the third display section is a touchscreen that allows theauthorized passenger to select the displayed alternative vehicleactions.
 10. The visual display of claim 1, further comprising: a knobthat allows the authorized passenger to select the displayed alternativevehicle actions.
 11. The visual display of claim 1, further comprising:a voice input that allows the authorized passenger to select thedisplayed alternative vehicle actions via voice command.
 12. The visualdisplay of claim 1, further comprising: a biometric sensor to collectinformation from passengers to determine if a passenger is an authorizedpassenger.
 13. The visual display of claim 12, wherein the biometricsensor comprises one of a fingerprint sensor, a handprint sensor, aretina sensor, facial recognition, an optical sensor, an ear shapesensor, or a voice sensor.
 14. The visual display of claim 1, wherein apassenger is identified as an authorized passenger based on one of averbal passcode, a hand-entered passcode, or a passenger inserts thevehicle key.
 15. The visual display of claim 1, further comprising: anartificial intelligence module that learns the alternative vehicleactions the authorized passenger selects in various interventionsituations, wherein the artificial intelligence module uses the learnedalternative vehicle actions to display commonly selected alternativevehicle actions to the authorized passenger for similar futuresituations.
 16. The visual display of claim 15, wherein the artificialintelligence module alters the vehicle actions taken by the autonomousvehicle based on the commonly selected alternative vehicle actionsselected by the authorized passenger.
 17. A method for controlling anautonomous vehicle, comprising: displaying an external view of theautonomous vehicle; displaying vehicle actions that the autonomousvehicle will take; displaying alternative vehicle actions that theautonomous vehicle may take; receiving a selection of the alternativevehicle actions from an authorized passenger of the autonomous vehicle;and overriding the vehicle actions with the selected alternative vehicleactions.
 18. The method of claim 17, wherein displaying the externalview includes identifying potential hazardous external obstacles anddisplaying overlaid simulated navigation directions based on the vehicleactions.
 19. The method of claim 17, wherein the selection is receivedby one of a touchscreen input, a knob input, a button input or a voicecommand.
 20. The method of claim 17, wherein the alternative vehicleactions comprise one of slowing down, increasing speed, maintaining thecurrent speed, stopping, altering a course of the vehicle or making aturn.
 21. The method of claim 17, further comprising: receiving abiometric input from a passenger of the vehicle; and identifying thepassenger as an authorized passenger based on the biometric input. 22.The method of claim 21, wherein the biometric input is received via abiometric sensor that comprises one of a fingerprint sensor, a handprintsensor, a retina sensor, facial recognition, an optical sensor, an earshape sensor, or a voice sensor.
 23. The method of claim 17, furthercomprising: receiving an input from a passenger of the vehicle, whereinthe input is one of a verbal passcode, a hand-entered passcode, or aninsertion of a vehicle key; and identifying the passenger as anauthorized passenger based on the input.
 24. The method of claim 17,further comprising: learning the alternative vehicle actions selected bythe authorized passenger in various intervention situations, wherein thedisplaying of the alternative vehicle actions is based on the previouslyselected alternative vehicle actions by the authorized passenger
 25. Themethod of claim 17 further comprising: altering the vehicle actionstaken by the autonomous vehicle based on the selected alternativevehicle actions selected by the authorized passenger.
 26. A visualdisplay for an autonomous vehicle, comprising: a first display sectionthat displays vehicle actions that the autonomous vehicle will take; anda second display section that displays alternative vehicle actions thatan authorized passenger of the autonomous vehicle may select to overridethe vehicle actions.
 27. The visual display of claim 26, wherein thealternative vehicle actions comprise one of slowing down, increasingspeed, maintaining the current speed, stopping, altering a course of thevehicle or making a turn.
 28. The visual display of claim 26, furthercomprising: a voice input that allows the authorized passenger to selectthe displayed alternative vehicle actions via voice command.
 29. Thevisual display of claim 26, further comprising: an artificialintelligence module that learns the alternative vehicle actions theauthorized passenger selects in various intervention situations, whereinthe artificial intelligence module uses the learned alternative vehicleactions to display commonly selected alternative vehicle actions to theauthorized passenger for similar future situations.
 30. A decisionsystem for an autonomous vehicle, comprising: an input sensor to collectinformation from passengers to determine if a passenger is an authorizedpassenger; and a display section that displays alternative vehicleactions that an authorized passenger of the autonomous vehicle mayselect to override vehicle actions of the autonomous vehicle.